Gradient-Based Neural Architecture Search: A Comprehensive Evaluation

One of the challenges in deep learning involves discovering optimal architecture for a specific task. This is effectively tackled through Neural Architecture Search (NAS). encompasses three prominent approaches—reinforcement learning, evolutionary algorithms, and gradient descent—that have demonstrated noteworthy potential identifying good candidate architectures. However, approaches based on reinforcement algorithms often necessitate extensive computational resources, requiring hundreds GPU days or more. Therefore, we confine this work to gradient-based approach due its lower resource demands. Our objective NAS method pinpointing opportunities future enhancements. To achieve this, comprehensive evaluation use four major Gradient descent-based search methods best neural image classification tasks provided. An overview these methods, i.e., DARTS, PDARTS, Fair DARTS Att-DARTS, presented. A theoretical comparison, spaces, continuous relaxation strategy bi-level optimization, deriving then The strong weak features are also listed. Experimental results comparing error rate cost analyzed. These experiments involved using bench marking datasets CIFAR-10, CIFAR-100 ImageNet. show that PDARTS better faster among examined making it potent automating Search. By conducting comparative analysis, our research provides valuable insights directions address criticism gaps literature.